Papers with self-supervised manner
Pretraining and Finetuning Language Models on Geospatial Networks for Accurate Address Matching (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing approaches to address matching and building authoritative address catalogues are limited in data quality and require labeling effort to develop accurate models. |
| Approach: | They propose to view addresses as an address graph and curate inputs by placing geospatially linked addresses in the same context. |
| Outcome: | The proposed framework improves address matching and fine-tuning language models. |
Language Model Pre-Training with Sparse Latent Typing (2022.emnlp-main)
Copied to clipboard
| Challenge: | Modern large-scale Pre-trained Language Models focus on text reconstruction, but have not sought to learn latent-level interpretable representations of sentences. |
| Approach: | They propose a new pre-training objective that enables the model to learn latent types . the objective allows the model a self-supervised way to extract sentence-level keywords . |
| Outcome: | The proposed model learns interpretable latent type categories without external knowledge and improves downstream tasks. |
Learning Reasoning Patterns for Relational Triple Extraction with Mutual Generation of Text and Graph (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods focused on learning text patterns from explicit mentions but failed to extract the implicitly implied triples. |
| Approach: | They propose to construct a relational graph from a sentence and apply multi-layer graph convolutions to capture the type inference logic of the paths. |
| Outcome: | The proposed framework can find multi-hop reasoning paths and capture type inference logic with the sentence's supplementary relational expressions. |